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(57) Abstract 

The present invention relates generally to a method and system for routing control in communication networks and for system control. 
More particularly, the present invention performs routing by controlling the components in a network with software agents (102) operating 
in a reward framework using p, tau, and patches (104) to improve communication performance (106). This invention disclosure includes 
the combination of reinforcement learning agents in a market-based or performance-based reward framework together with optimization 
techniques called p, tau, and patches (104) as applied to the problem of topology-and load-based routing in data networks, in order to 
improve communication performance (106) such as communication latency and bandwidth. The invention also applies to the control of 
other systems, including operations management, job-shop problems, organizational structure, portfolio management, risk management etc. 



FOR THE PURPOSES OF INFORMATION ONLY 



Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under the PCT. 



AL 


Albania 


ES 


Spain 


LS 


AM 


Armenia 


FI 


Finland 


LT 


AT 


Austria 


FR 


France 


LU 


AU 


Australia 


GA 


Gabon 


LV 


AZ 


Azerbaijan 


GB 


United Kingdom 


MC 


BA 


Bosnia and Herzegovina 


GE 


Georgia 


MD 


BB 


Barbados 


GH 


Ghana 


MG 


BE 


Belgium 


GN 


Guinea 


MK 


BF 


Burkina Faso 


GR 


Greece 




BG 


Bulgaria 


HU 


Hungary 


ML 


BJ 


Benin 


IE 


Ireland 


MN 


BR 


Brazil 


IL 


Israel 


MR 


BY 


Belarus 


IS 


Iceland 


MW 


CA 


Canada 


IT 


Italy 


MX 


CF 


Central African Republic 


JP 


Japan 


NE 


CG 


Congo 


KE 


Kenya 


NL 


CH 


Switzerland 


KG 


Kyrgyzstan 


NO 


CI 


Cote d'Tvoire 


KP 


Democratic People's 


NZ 


CM 


Cameroon 




Republic of Korea 


PL 


CN 


China 


KR 


Republic of Korea 


FT 


cu 


Cuba 


KZ 


Kazakstan 


RO 


cz 


Czech Republic 


LC 


Saint Lucia 


RU 


DE 


Germany 


U 


Liechtenstein 


SD 


DK 


Denmark 


LK 


Sri Lanka 


SE 


EE 


Estonia 


LR 


Liberia 


SG 



Lesotho 

Lithuania 

Luxembourg 

Latvia 

Monaco 

Republic of Moldova 

Madagascar 

The former Yugoslav 

Republic of Macedonia 

Mali 

Mongolia 

Mauritania 

Malawi 

Mexico 

Niger 

Netherlands 

Norway 

New Zealand 

Poland 

Portugal 

Romania 

Russian Federation 

Sudan 

Sweden 

Singapore 



SI 


Slovenia 


SK 


Slovakia 


SN 


Senegal 


SZ 


Swaziland 


TD 


Chad 


TG 


Togo 


TJ 


Tajikistan 


TM 


Turkmenistan 


TR 


Turkey 


TT 


Trinidad and Tobago 


UA 


Ukraine 


UG 


Uganda 


US 


United States of America 


UZ 


Uzbekistan 


VN 


Viet Nam 


YU 


Yugoslavia 


ZW 


Zimbabwe 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US00/02011 



A. CLASSIFICATION OF SUBJECT MATTER 
1PC(7) :H04M 15/00; H04L 12/46; G06F 15/18 
US CL : Please See Extra Sheet. 

According to International Patent Classification (IPC) or to both national classification and IPC 

B. FIELDS SEARCHED 

Minimum documentation searched (classification system followed by classification symbols) 
U.S. : 370/252, 253. 255, 270, 400, 408; 70678, 12. 19, 25; 709/238, 242, 243 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 


C. DOCUMENTS CONSIDERED TO BE RELEVANT 


Category* 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


X,E 
Y 


US 6,046,985 A (ALDRED ET AL) 04 APRIL 2000, col. 4, line 59 
to col. 11, line 39. 


1, 2, 20, 21, 26, 
28 






3-16, 27, 47-48 


Y 


US 5,790,642 A (TAYLOR ET AL) 04 AUGUST 1998, col. 4, line 
11 to col. 9, line 53. 


3-16, 27, 47-48 


A 


AHN ET AL, GENROUTER: A GENETIC ALGORITHM FOR 
CHANNEL ROUTING PROBLEMS, IEEE, MAY 1995, pp. 151- 
154. 


1-50 


| | Further documents are listed in the continuation of Box C. | | Sec patent family annex. 


" Special categories of cued documents: 

- A" document defining the general state of the art which is not considered 
to be of particular relevance 

*E" earlier document published on or after the international filing date 

*L* document which may throw doubts oti priority claim (s) or which is 
cited to establish the publication date of another citation or other 
special reason (as specified) 

*0* document re ferritin in un ort*l disclosure, use. exhibition or other 
means 

•P" document published prior to the international filing date but later titan 
the priority date claimed 


"T" later document published after the international filing data or priority 
data and not in conflict with the application but cited to understand 
tho principle or theory underlying the invention 

"X* document of particular relevance; the claimed invention cannot be 
considered novel or cannot be considered to involve an inventive step 
when the document is taken alone 

*Y" document of particular relevance; the claimed invention cannot be 
considered to involve an inventive step when the document is 
combined with one or more other such documents, such combination 
being obvious to • person skilled in the art 

document member of the same patent family 


Date of the actual completion of the international search 
07 JUNE 2000 


Date of mailing of the international search report 

06 JUL 2000 


Name and mailing address of the ISA/US 
Commissioner of Patents and Trademarks 
Box PCT 

Washington, D.C. 20231 
Facsimile No. (703) 305-3230 


Authorized officer ^ M a . 
DUONG, FRANK VjfirfC HjJL* 

Telephone No. (703) 308-5428 



Form PCT/ISA/210 (second sheet) (July 1998)* 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US00/0201 1 



A. CLASSIFICATION OF SUBJECT MATTER: 
USCL : 



370/252, 253, 255. 270, 400, 408; 706/8, 12, 19 , 25; 709/238, 242, 243 



Form PCT/ISA/210 (extra sheet) (July 1998)* 



PATENT COOPERATION TREATY 

FILE COPY 409 

INTERNATIONAL PRELIMINARY EXAMINATION REPORT 
(PCT Article 36 and Rule 70) 

Form PCT/IPE A/409 (cover sheet) (July 1998) FILE COPY - DO NOT MAIL 




Applicant's or agent's file reference 
9392-023-228 


FOR FURTHER ACTION Sec Notification of Transmittal of International 

Preliminary Examination Report (Form PCT/IPEA/4 16) 


International application No. 
PCT/US00/020H 


International filing date (day/month/year) 
28 JANUARY 2000 


Priority date (day/month/year) 
28 JANUARY 1999 


International Patent Classification (IPC) or national classification and IPC 
Please See Supplemental Sheet. 


Applicant 
BIOS GROUP LP 



This international preliminary examination report has been prepared by this International Preliminary 
Examining Authority and is transmitted to^the applicant according to Article 36. 



2. This REPORT consists of a total of 



sheets. 



| | This report is also accompanied by ANNEXES, i.e., sheets of the description, claims and/or drawings which have 
been amended and are the basis for this report and/or sheets containing rectifications made before this Authority, 
(see Rule 70.16 and Section 607 of the Adrninistrative Instructions under the PCT). 

These annexes consist of a total of ^) sheets. 



3. This report contains indications relating to the following items: 
Basis of the report 
Priority 

Non-establishment of report with regard to novelty, inventive step or industrial applicability 
Lack of unity of invention 

Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applical 
citations and explanations supporting such statement 



I 


[1 


II 


□ 


III 


□ 


IV 


□ 


V 




VI 


□ 


VII 


□ 


VIII 


□ 



V 



Date of submission of the demand 
21 AUGUST 2000 


Date of completion of this report 
15 DECEMBER 2000 


Facsimile No. 

(703) 305-3230 


Authorized^bfflcer/s^D Teteph0tf£>f6 - r? 



INTERNATIONAL PRELIMINARY EXAMINATION REPORT 
Form PCT/IPEA/409 (Box I) (July 1998) 
FILE COPY - DO NOT MAIL 



International application No. 
PCT/US00/02011 



L Basis of the report 



1. Wilh regard to the elements of the international application:* 
| x | the international application as originally filed 
the description: 



pages , 
pages , 
pages . 



1-20 



NONE 



_ , as originally filed 
filed with the demand 



NONE 



filed with the letter of 



E 



the claims: 

pages 

pages 

pages 

pages 



21-30 



NONE 



NONE 



, as originally filed 

, as amended (together with any statement) under Article 19 
, filed with the demand 



NONE 



filed with the letter of 



| x| the drawing: 

pages 

pages 

pages 



1-2 



NONE 



, as originally filed 

, filed with the demand 



NONE 



I x| the sequence listing part of the description: 
1 — 1 NONE 



, filed with the letter of . 



pages 
pages 
pages 



NONE 



, as originally filed 

, filed with the demand 



NONE 



filed with the letter of 



2. With regard to the language, all the elements marked above were available or furnished to this Authority in the language in which 
the international application was filed, unless otherwise indicated under this item. 

These elements were available or furnished to this Authority in the following language which is: 

| | the language of a translation furnished for the purposes of international search (under Rule 23.1(b)). 
| | the language of publication of the international application (under Rule 48.3(b)). 

| | the language of the translation furnished for the purposes of international preliminary examination (under Rules 55.2 and/ 
or 55.3). 

3. With regard to any nucleotide and/or amino acid sequence disclosed in the international application, the international 
preliminary examination was carried out on the basis of the sequence listing: 

□ contained in the international application in printed form. 

[ | filed together with the international application in computer readable form. 

[ [ furnished subsequently to this Authority in written form. 

| | furnished subsequently to this Authority in computer readable form. 

□ The statement that the subsequently furnished written sequence listing does not go beyond the disclosure in the 
international application as filed has been furnished. 

I I The statement that the information recorded in computer readable form is identical to the writen sequence listing has 
1 — 1 been furnished 

4 | x[ The amendments have resulted in the cancellation of: 

03 the description, pages NONE 

the claims, Nos. 



NONE 



fxl the drawings, sheets/fig NONE 



5. I I This report has been drawn as if (some of) the amendments had not been made, since they have been considered to go 

*— J beyond the disclosure as filed, as indicated in the Supplemental Box (Rule 70.2(c)).** 
* Replacement sheets which have been furnished to the receiving Office in response to an invitation under Article 14 are referred to 
in this report as "originally filed" and are not annexed to this report since they do not contain amendments (Rules 70.16 and 70.17). 




INTERNATIONAL PRELIMINARY EXAMINATION REPORT 


International application No. 




Form PCT/IPE A/409 (Box V) (July 1998) 




FILE COPY - DO NOT MAIL 


PCT/USOO/02011 





V. Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 



1 . statement 



Novelty (N) Claims 1-50 : YES 

Claims NONE NO 

Inventive Step (IS) Claims i^O YES 

Claims NONE NO 

Industrial Applicability (IA) Claims _L5? YES 

Claims NONE NO 



2. citations and explanations (Rule 70.7) 

Claims 1-50 meet novelty under PCT Article 33(2) because the prior art of record, considered individually or in combination, 
fails to show or suggest a method and system for routing control in communication networks that performs routing by controlling 
the components in a network with software agents operating in a reward framework, wherein a fitness landscape optimization 
algorithm is used to improve communication latency and bandwidth in a manner set forth as claimed. Moreover, the cited 
references of record are relevant to the claimed invention, but fails to establish themselves as the prior art 

Claims 1-50 meet industrial applicability as defined by PCT Article 33(4) since the instant invention is applicable in routing 
control of a communication network. 



NEW CITATIONS 

NONE 



INTERNATIONAL PRELIMINARY EXAMINATION REPORT 
Form PCT/IPE A/409 (Supplemental Box) (July 1998) 
FILE COPY - DO NOT MAIL 



International application No. 
PCT/USOO/02011 



Supplemental Box «- • x 

(To be used when the space in any of the preceding boxes is not sufficient) 



Continuation of: Boxes I - VIII 



Sheet 10 



CLASSIFICATION: . 

The International Patent Classification (IPC) and/or the National classification are as listed below: 
IPC(7): H04M 15/00; H04L 12/46; G06F 15/18 and US CL: 370/252, 253, 255, 270. 400, 408; 706/8, 12, 19, 25; 709/238, 
242, 243 



